{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Application of batch-mode regression to cross-validation\n",
    "\n",
    "botorch provides a helper function `gen_loo_cv_folds` to easily perform leave-one-out (LOO) cross-validation (CV) by taking advantage of batch-mode regression and evaluation in GPyTorch. This tutorial illustrates the process on a noisy sinusoidal function, similar to the example from the batch-mode GP regression [tutorial](https://github.com/cornellius-gp/gpytorch/blob/master/examples/01_Simple_GP_Regression/Simple_Batch_Mode_GP_Regression.ipynb) from GPyTorch:\n",
    "\n",
    "$$y = \\sin(2\\pi x) + \\epsilon, ~\\epsilon \\sim \\mathcal N(0, 0.2).$$\n",
    "\n",
    "Note: this tutorial aims to introduce batch-mode regression and evaluation in GPyTorch with CV as an example application. For alternative, more user-friendly functions to perform CV in Ax, see [ax.modelbridge.cross_validation](https://github.com/facebook/Ax/blob/master/ax/modelbridge/cross_validation.py). However, for larger CV tasks, it may be useful to exploit GPyTorch batch-mode, as shown in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import math\n",
    "\n",
    "device = torch.device(\"cpu\")\n",
    "dtype = torch.float\n",
    "torch.manual_seed(3);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize the CV dataset\n",
    "\n",
    "For our training data, we take 20 regularly spaced points on the interval $[0, 1]$ and generate noisy evaluations with an observed noise variance of 0.2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "sigma = math.sqrt(0.2)\n",
    "train_X = torch.linspace(0, 1, 20, dtype=dtype, device=device).view(-1, 1)\n",
    "train_Y_noiseless = torch.sin(train_X * (2 * math.pi)).view(-1)\n",
    "train_Y = train_Y_noiseless + sigma * torch.randn_like(train_Y_noiseless)\n",
    "train_Yvar = torch.full_like(train_Y, 0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The botorch function `gen_loo_cv_folds` takes our observed data `train_X`, `train_Y`, `train_Yvar` as input and returns the LOO CV folds in a `CVFolds` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from botorch.cross_validation import gen_loo_cv_folds\n",
    "\n",
    "cv_folds = gen_loo_cv_folds(train_X=train_X, train_Y=train_Y, train_Yvar=train_Yvar)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `cv_folds` object contains the data, stored as tensors of appropriate batch shape, necessary to perform 20 CVs of 19 training points and 1 test point. For example, we can check that the shapes of the training inputs and training targets are `b x n x d = 20 x 19 x 1` and `b x n x o = 20 x 19 x 1` respectively, where `o` is the number of outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([20, 19, 1]), torch.Size([20, 19, 1]))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv_folds.train_X.shape, cv_folds.train_Y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([20, 1, 1]), torch.Size([20, 1, 1]))"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv_folds.test_X.shape, cv_folds.test_Y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that in a situation where the dataset is large, one may not want to perform LOO; in that case, a similar process can be used to perform $k$-fold CV."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Perform LOOCV\n",
    "\n",
    "We can the `batch_cross_validation` function to perform LOOCV using batching (meaning that the `b = 20` sets of training data can be fit as `b = 20` separate GP models with separate hyperparameters in parallel through GPyTorch) and return a CVResult tuple with the batched `GPyTorchPosterior` object over the LOOCV test points and the observed targets. The `batch_cross_validation` requires a model class (`model_cls`) and a marginal log likelihood class (`mll_cls`). Since we have an observed and constant noise level, we will use the FixedNoiseGP as the `model_cls` and an ExactMarginalLogLikelihood as the `mll_cls`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from botorch.cross_validation import batch_cross_validation\n",
    "from botorch.models import FixedNoiseGP\n",
    "from gpytorch.mlls.exact_marginal_log_likelihood import ExactMarginalLogLikelihood\n",
    "\n",
    "# instantiate and fit model\n",
    "cv_results = batch_cross_validation(\n",
    "    model_cls=FixedNoiseGP,\n",
    "    mll_cls=ExactMarginalLogLikelihood,\n",
    "    cv_folds=cv_folds,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Compute the cross-validation error and generate plots\n",
    "To compute the cross-validation error, we first evaluate the test points by computing the posterior in batch mode. Next, we compute the squared errors for each test point from the prediction and take an average across all cross-validation folds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cross-validation error:  0.14\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXxU1fnH8c+TkAQCQUH2JYJ7FS1VlE2qFTcClkVlUShYKbhV3LBaf9WqKC5ttS4toEWKKyAiIFGLiiJIWBQVQVFkkU3ZZAnZk/P7IxOMIZmZJDNzZ5Lv+/XKi8zcM/c+M5p57r3nnOeYcw4REZGKxHkdgIiIRDclChER8UuJQkRE/FKiEBERv5QoRETErzpeBxAOTZo0ce3atfM6DBGRmPHxxx/vcs41LW9bjUwU7dq1Y8WKFV6HISISM8xsU0XbdOtJRET8UqIQERG/lChERMQvJQoREfFLiUJERPxSohAREb+UKERExC8lChER8UuJQkT8GjRxCYMmLvE6DPGQEoWIRBUlpuijRCEiIn4pUYiIiF9KFCIi4pcShYiI+KVEISIifnmaKMxsspntMLMvKth+rpntM7NPfT93RzpGEZHazuuFi6YATwFT/bT50DnXJzLhiEhN45zDzLwOI6Z5ekXhnFsI7PEyBhGpub799lu6dOnC6tWrvQ4lpsVCH0VXM/vMzN40s1MqamRmo8xshZmt2LlzZyTjE5EolJGRQZcuXVi2bBl33HGH1+HENK9vPQXyCXC0cy7TzNKA14Hjy2vonJsETALo1KmTi1yIIhItSmZ0D2yyjaFDh5KTk8NFF13Eiy++6HFksS2qryicc/udc5m+39OBBDNr4nFYIhKlnHOsfedlLr/8cnJychg5ciRz586lYcOGXocW06I6UZhZC/P1QpnZWRTHu9vbqERqp2ivwVRQUMDKV/7BZ68+iXOO8ePHM2nSJBISErwOLeZ5euvJzF4GzgWamNkW4B4gAcA5NwG4DLjWzAqAbGCwc063lUTCoCQJTBvd1eNIKi8zM5MhQ4aw7oM3iKuTwIvPT2Xw4MFeh1VjeJoonHNDAmx/iuLhsyIi5dq+fTt9+vThk08+IbF+Q7pf85CSRIhFe2e2iEiFVq9eTVpaGt999x3HHHMMJ/zuARq2ONrrsGqcqO6jEBGpyLvvvku3bt347rvv6NKlCxkZGUoSYaJEISIxZ8qUKVx88cXs37+fSy+9lPfee4+mTZt6HVaNpUQhIjHDOcc999zDVVddRUFBAbfddhvTp0+nXr16XodWo6mPQiQIsTwiqKbIy8tj5MiRPP/888TFxfHkk09y3XXXeR1WraBEISJR78cff2TAgAG8//77JCcnM23aNPr0Ua3QSFGiEJGotnHjRtLS0vjyyy9p0aIFb7zxBmeccYbXYdUq6qMQ8VC0z3b22vLly+ncuTNffvklp5xyCkuXLlWS8IAShYhEpdmzZ3POOeewY8cOevbsyeLFi0lNTfU6rFpJiUJEos7X702nf//+ZGdnM2LECNLT0zniiCO8DqvWUqIQkahRWFjIyumP8+n0x3HOcd999zF58mQSExO9Dq1WU2e2iESFrKwsrrzySr5573Xi4uvw3ynPMXToUK/DEpQoRCQK/PDDD1xyySUsX76chOQUuo8eryQRRZQoRMRTX375JWlpaWzcuJF27dpx0vAHadiynddhSSnqoxARz3zwwQd069aNjRs3cuaZZxYX9lOSiDpKFCLiiRdffJELLriAvXv30rdvX95//32aN2/udVhSDiUKkVqkKhP88gqKWLNtP3kFRSGJwTnHuHHjGDp0KPn5+YwZM4aZM2eSnJwckv1L6ClRiIhfW/dmcyC3gK17s6u9r/z8fEaOHMlf/vIXzIzHH3+cxx9/nPj4+ENtShLTjgM51T6ehIbVxCWoO3Xq5FasWOF1GFKDDJq4hDXb93Nyy4Yh3e+a7fsBQr7fqhyv7LZlG/dQ3teDGZzVrnGlj52XncmSiXfxw1fLiU9IosvV99K6468Pa/fJdz+SX+holpJE+yb1K3WMcHyetaVisJl97JzrVN42jXoSiVIlX3qhlJVbUOG+y25LTownN7+IgqKfZ4vkxPhKx5a3bwebXrqb3B0bia9/JEcPuZd9zU9kX6n9HMgp+NlrdhzIZceBXABS6gb3VeXv/VVVpGtxRWNiUqIQCdLJLRuG/I/Y3zoX4fiCqswVBcCGXZnsOJB36HFCvNGhVeVKafz43Vo+fG4suft20bBFO86+4W80aNLqsHZ5BUV8tyeL3QeLjxdn0Cg5kdTGySTWCe4ueaSv0GoLJQqRKBWOM8tgElPpbaOfX8HK7/bSLCWJHQdyyS8sqlRc8+bNY9CtN5Bz8CDnnnsur732Go0aNaqw/V2zVvHi0u8wAwf06tCCcf1PDfp4WmAqPDztzDazyWa2w8y+qGC7mdkTZrbOzD43s9MjHaNIbTZxWCfaN6lP/aQ6tG9SnxOapwT92n//+9/89re/5eDBgwwdOpS33nrLb5IA2JWZS7OUJE5p2ZArOx/Nzszc6r4FCQGvRz1NAS72s70XcLzvZxTw7wjEJCLVUFRUxNixY7nuuusoKiri7rvvZurUqSQlJQV8benENK5fByYOK7dvVSLM01tPzrmFZtbOT5O+wFRXPDQrw8yONLOWzrntEQlQRColOzub3/3ud7z66qvUqVOHZ555hhEjRngdllRTtPdRtAY2l3q8xfecEoVIlNm5cyd9+/ZlyZIlNGzYkNdee42ePXt6HZaEQLQniqCZ2SiKb09pFSyRCPv6669JS0vj22+/JTU1lXnz5tGhQwevw5IQ8bqPIpCtQNtSj9v4njuMc26Sc66Tc65T06ZNIxKciMCiRYvo2rUr3377LaeffjoZGRmeJQnN6g6PaE8Uc4Df+UY/dQH2qX9CJHhVqe1UGdOmTaNnz57s2bOH3r1788EHH9CyZcuwHS+QknIjT7zzjWcx1ESe3noys5eBc4EmZrYFuAdIAHDOTQDSgTRgHZAFXOVNpCKxpSQ5lExAq+hxaRVtKzvLedDEJTjn+Op/L7BqVvFAxGPPGUC9tJu4+sVV1Y7dX4wVKVtu5IWl3/GCbz5GVcqNxKpwzR/xetTTkADbHXB9hMIRqdWKnCMnr5C8gqJyZ0KXbM/Ny2PVjMdY/+FsAH556Q2ccP4QzCzSIR/Ssc2RFc7qluqrMZ3ZIvKTkjPLsjOV/c1cPuuBdziYW8gprRr+bDZ0yWs27DrIwdxCNn62hPUfzqZu3bo8//zzXHbZZSGNvaqzq6s7q1sqpuqxImX4uy1zcsuGIS04V1LELjkpPOdsZfdf3vHKFuMrLaVunYq3uyJS6iWGKNKfVPUzyc4rpKDIEQfExxvOQb3E+ICvKxGoPlRNLwvir3pstHdmi0iY1U+Kp07cz28b1Ykz6icVf8nGAUU5ByjKLy6nUZSfixVkU79uQqRD9ateYjzxVlwGvW5CfKWShPinW08iZQRbMC8UKrvf6rav6PWlb9sADD6z7aHbNufc+Bir122kfofzcYX5xCckcWXn1LDd1qnOZ62igOGhKwoRqbAY3zPPPMOHT4/FkhrAtlWc0jKFK7uoWF9toysKkRgVyrPnicM6HdrfuH4dKCoq4s9//jPjx48HoEnOFk69cBQWF8e4fppxXdsoUYjIz+Tk5HDVVVfxyiuvEB8fT8fBt3Fsj75ehyUeUqIQkUNyM/dxwQUXsGjRIho0aMCrr77K5I1aLa62Ux+FiIeiqTZR5s4tvPvIKBYtWkTr1q1ZtGgRF110kddhSRRQohDxULTUJlqyZAnvPjyKzB2bOe2008jIyOCXv/ylpzFJ9NCEO/FcOIvWhUrpCXehULY2UYlAtYnKm/jnL6aybcp7zZZPFrD0uXspzM+jwbFncMEND5FQr/5h+ygRqs8AQj8UWcNjq87fhDv1UYh4IBpqEznn+Pqdl/nstafBORqd3otWadf/LEmIgBKFRIFYOPsLx5lqVWoTlY4jmJgqmnD34tVnMmbMGGbM/BcADz30EB8f0QMzO2x/Za/4YuG/l4SWEoWIR0omuTVLSaJjaiN2RqhDOz8ni379+jFv3jwSExOZOnUqgwYNiolbgOINJQoRj5Sd5BYJ2Xt38uHTY9m7+WsaN27M7NmzOfvssyNybIldShQitcQXX3zBuw+PIuvHHzj22GNJT0/nhBNO8DosiQEaHitSC7zzzjt0796drB9/4KhjOrBkyRIlCQmaEoVIDbfhozfo1asX+/fvp02XPrS48hFc3RSvw5IYokQhUkM55/hizjMsn/ogBQUFjB07lpa9byQzr8jzCX4SWzThTmqlkk7kYCfShXrCXVX3G+yEu8L8PFa8MJ5NS98Gi+Po22ZB3OEL+ZSe4FfR/sI54a481fmsA71WQ3srpgl3IrVI3sH9LJ74Z3Z+/QlxCXVpe9md/Oroozyf4CexS4lCaqVAK76VFa6yEtVZsa68127YsIG0tN+z8+uvaNmyJR2uepBGqScybXTXgBP8Kool0hPuVMIj+qiPQqSGWLZsGV26dOGrr76iQ4cOZGRk0Cj1xEPbK1rFTiQQTxOFmV1sZmvNbJ2Z3VHO9hFmttPMPvX9jPQiTpFo9/rrr3PuueeyY8cOzj//fBYtWkRqaurP2kwc1on2TepTP6kO4/p1YOKwcm9HH2ba6K46Q6/lPLv1ZGbxwNPABcAWYLmZzXHOrSnTdJpz7oaIBygSI/75z39y880345zjqquuYuLEiSQkJHgdVpUpKUUfL/sozgLWOefWA5jZK0BfoGyiEJFyFBUV8tmMJ5i+YAYA999/P3fddRdm5nFkUtN4mShaA5tLPd4CdC6n3aVm9mvga+Bm59zmctpgZqOAUcBhl9wS29RBebiDBw/y0cQ/s+2zD0lISGDy5MkMHTrU67Ckhor2zuy5QDvn3GnAfOC/FTV0zk1yznVyznVq2rRpxAIUibSc/Xv4zW9+U5wkklOYP3++koSElZdXFFuBtqUet/E9d4hzbneph88Cj0QgLpGotX/7Rj586lYO7t5O/aNa0uOGv3POOed4HZbUcF4miuXA8WbWnuIEMRi4onQDM2vpnNvue/hb4MvIhigSPRYsWMC7j44mP+sAZ511Fi0uu5u6DSteNlUkVDy79eScKwBuAN6mOAFMd86tNrP7zOy3vmY3mtlqM/sMuBEY4U20It56/vnnueiii8jPOkDrX/6aBQsWKElIxHg6M9s5lw6kl3nu7lK/3wncGem4RKKFc47777+fe+65B4DjzxvELy+7geRkld6QyPGbKMzsFn/bnXP/CG04IlIiLy+P0aNHM2XKFMyMxx9/nMVJZ3odltRCgW49pfh+OgHXUjyktTVwDXB6eEMTqb327t1LWloaU6ZMoV69esyaNYsbb7zR67CklvJ7ReGcuxfAzBYCpzvnDvge/xWYF/boRGqhTZs20bt3b1avXk3z5s2ZO3cuZ56pK4lgaK5NeATbR9EcyCv1OM/3nIiE0Mcff0yfPn34/vvv+cUvfkF6ejrt2rU7tD2voIh1OzLZcSDHuyCl1gk2UUwFlpnZLN/jfviZ/CYilffGG28waNAgsrKyOPfcc3nttddo1KjRz9ps3ZvNgdwCrVAnERX0CndmdjrQw/dwoXNuZdiiqiatcFezlFfCo+waCVUViRXu/L22ZFvi2vmsnPYYzhVxdOeL6TTsTuLr/FTYb9nGPVT0p9q5fcXDZMseO5yrx0WCbi2FT6hWuEsG9jvnnjOzpmbW3jm3ITQhitROzhXx/fz/sHvJTABO7v17Tulz9WGF/Tq2OfKwFerizEhKiNxUKC8ThHgrqERhZvdQPPLpROA5IAF4AegevtBEKhaqM0svV7jLzs7mhO5p7F75PnXq1OHZZ59l+PDhFe6n7Ap1jesn0r5Jfb8xlT22CixKVQR7OtKf4hIaBwGcc9soHjYrIlWwY8cOzjvvPLasfJ+Eeg146623/CYJOHyFutyCQtZs26+ObQm7YBNFnivuzHAAZlY/fCGJRJ+8gqKQfSmvXbuWrl27kpGRQXLj5pw3dgI9e/YM+LqyK9Ql1YlXx7ZERFCd2WZ2G3A8xavRjQd+D7zsnHsivOFVjTqza4ayHdYV3S6pTsd2sB20n3z3I/mFjmYpSbRvUrnzpNLH2PnNpyyecAd5B/fTKPVEml12NwkpRwV9/3/N9v0cyCkod5sZnNWu8WHtS45d3uNYp1tooeOvM7syo54uAC4EDHjbOTc/dCGGlhJF+IRqtFEwSr7USlT05Va2XWVk5RZ/6SYnld9dV9GXMkBK3eDGgpQcI++bRWx9/e+4wnxSTuhMm0vvINcl+D1+eftyDuLijIKin/5268QVd2zHlekEL/v+Ar3fikRrYlGiCJ1qj3oys4edc3+iePGgss+JeKo6X2KBzrDzCooOG23UKDmR1MbJJNYJ7s7t6m372LV4Oj+8+xwAx51zKR0H3URcXHylz/BL2tdLiGPHgZ/mwJZ0bAd6f1W9otAXcu0W7GnFBUDZpNCrnOekhovkF0awt55CcQx/+y472qhXhxaM639qUPvPz8/npPMu44dFczAz/v73v3PTTTcdGv5a2VFIJe2PTE5g5Xd7aZaSRMfURuw8kMPEYYefDGrUk4RCoOqx1wLXAcea2eelNqUAH4UzMJFoUTLaqPSXcjD279/PwIEDWb/obeITEpn+yssMGDAgJDFNHNbp0Jf+uH4dQrJPkYoEuqJ4CXiT4g7sO0o9f8A5tydsUUmN4uVZbCiOXZUv5S1bttC7d28+//xzklKO5OxrHyk3SejMXmJBoOqx+4B9ZvZPYE+p6rENzayzc25pJIIUiSWffvopvXv3Ztu2bZxwwgkcN2wcDZq28ToskSoLdh7Fv4HMUo8zfc+JSClvvfUWPXr0YNu2bfTo0YMlS5YoSUjMCzZRmCs1jtY5V4THy6iKRJtJkybRp08fMjMzGTJkCPPnz6dxY61rLbEv2ESx3sxuNLME388YYH04A5PIGjRxSUTnSNQkRUVF3HnnnYwePZrCwkLuuusuXnjhBZKSkrwOTSQkgr0quAZ4Avg/ikcIvguMCldQIrEiJyeHESNGMG3aNOLj45kwYQIjR470OiyRkAoqUTjndgCDwxyLSEzZvXs3ffv2ZfHixaSkpPDqq69y4YUXHtau9Kp0zVLqehCpSPX4vfVkZrf7/n3SzJ4o+1Pdg5vZxWa21szWmdkd5WxPMrNpvu1LzaxddY8pUlpVi/2tW7eOrl27snjxYtq0acOiRYvKTRKgVekk9vmt9WRmlzjn5ppZufWPnXNVXg7VzOKBryme9b0FWA4Mcc6tKdXmOuA059w1ZjYY6O+cGxRo36r1VHnhnOtQ1b6PYGs9BbOPil4bbLG/0vvZtX4Vi//1J3Iz93Jk2+M5+7q/kdyo6WGvqWhVuvKK9wWrdBzBlOOIhiKAmisSG0JSFDDUzKwr8Ffn3EW+x3cCOOfGl2rztq/NEjOrA3wPNHUBglaiqLxT//o2EJ4vlKoW7SspYFeisoXsSu+j7GsrW+yvZD/53y5hy2uP4ArzaXDcmbS97E7ik5LL3U+Rc+TmFwVVvC9Ypd9PMAX+ArWJRAJRoogNVS4KaGZz8a1BUR7n3G+rEVdrYHOpx1uAzhW1cc4VmNk+4ChgVzmxjsLXwZ6amlqNsCRc/K0Z7W+bv9cHUtH+K1vsb/W2fexeMpPv5z8LwDE9+nH64FuIi/efvDbsygyqeF9V3k8orij0JS7BCHSK9jffvwOAFhQvfwowBPghXEFVhXNuEjAJiq8oPA4n5pR8kUS68F4w20o6g5+84leV7gz2t/9gi/0VFBRwUs+BfL9wFgAPP/wwY8eOPWxd6/KMfn5FUMX7qvJ+grldqCKAEgqBSnh8AGBmfy9zSTLXzKp7b2cr0LbU4za+58prs8V36+kIYHc1jysxpnRncLBVW4MRTLG/zMxMBg0axLcL04mrk8jLLz7PwIEDgz6GivdJTRDsTd/6ZnaMc249gJm1B6q7HOpy4HjfvrZSPPz2ijJt5gDDgSXAZcB7gfonarJwTogruUURjmP427e/bUs3/Lzu5AtLv+MF3xVAsJ3Bgd5Xdn4hm/ZkUd93D790u+y9O/nw6bHs3fw18fUakjr4Hmb+2JaZlfyMqvvZ6mpAvBZsorgZeN/M1lO8wt3RwOjqHNjX53AD8DYQD0x2zq02s/uAFc65OcB/gOfNbB2wB83lqFXqJ8X/rDO4dD9CuO3d+i2LnrqNrB9/oEHTNrQcdC9JR7UO+3FFolGwE+7eMrPjgZN8T33lnMut7sGdc+lAepnn7i71ew5weXWPU1OE88wyEsNjq9JHUdIZXJVFgwLtv6Lt77zzDpfefj1Z+/fTrVs3Zs+ezfUzv/G7n+rEIBLtgqr1ZGbJwFjgBufcZ0CqmfUJa2QicGiOwyktG3Jl56PZmVnt8xO/Jk+eTK9evdi/fz+XX3457777Lk2aNAnrMUWiXbC3np4DPgZKTom2AjOAN8IRlEiJE5qnHPo9nJ3Bzjnuvvtuxo0bB8Dtt9/O+PHjiYsLtm5mdFL5EAmFYP8KjnXOPQLkAzjnsijuqxCJeYX5eQwbNoxx48YRFxfHv//9bx5++OGYTxKg8iESGsFeUeSZWT18k+/M7FggvPcAREKoohFHq9ZvZdO0+8jatIo6SfXoOvJ+FtgvWVCmfXVGLoVyRFnpffnbb9nyIVUZMVYTqF8oNIJNFPcAbwFtzexFoDswIlxBiURC5q5trJ98C7m7NlPviCacff2jNEo90euwQqJjmyMrnHkuUlkBE4UVTz/9iuLZ2V0ovuU0xjl3WBkNkWhV9sxy2bJlXHLJdeTu2sERrY9l1ZIFtG3btoJXV2/kUihHPVVmZnawM89FAgmYKJxzzszSnXOnAvMiEJNIWM2aNYsrr7yS7Oxsmv/iTLqNetBvkohVwcw8FwlGsLeePjGzM51zy8MajUiYPf7449xyyy0457j66qvZd/rwgIX9YpXKh0ioBDusozOQYWbfmtnnZrbKzD4PZ2AioVRYWMiYMWO4+eabcc7xwAMP8Mwzz9TYJCESSsH+lVwU1ihEwqggN5sBAwYwZ84cEhMTee6557jiirJlxUSkIoHWo6gLXAMcB6wC/uOcq3jFF5Eok71vN4v+NZYfN31Fo0aNeP311/n1r3/tdVhVpuGe4oVAVxT/pXiS3YdAL+BkYEy4gxIJhTVr1vDuw38ga8/3tG/fnjfffJMTT6wZw19FIilQojjZN9oJM/sPsCz8IYkUKyk/cVyzBuWuOufPggUL6N+/P1n79tG4/SlkZLxHs2bNwhRpMRX/k5oq0F9ffskvuuUkkVZSfmLr3uxKvW7q1KlcdNFF7Nu3j9Ydz+Hcm58Me5IQqcnM3zpAZlYIHCx5CNQDSuo8Oedc+Fdmr4JOnTq5FSuquwBf9AjngkUlgll/ORz7Lm9b2fITJQKVn3DOsSb9OVbPLV7X+oTzB5PQdRgWF1/h+wr2fVdnfeqqfraBrky0FKqEkpl9XGYl00MCLYUaH56QRCpWtvwEQJ0449TWR1T4msKCfD5+4WE2ZqRjFkfHgTdx/G8uO/QlLSJVp0HkMSASZ4PhPPPs//Ri1u3I5MkrfnVYqeuKjvuz8hMOGtdPZNb13cvd/969e7n00kvZmPEeycnJvPLKK1xyySUBj+3v+FVpV1EbndVLrIv9OsoS9apS6rqk/MQpLRvSLCWJ/MKicttt2rSJ7t27895779GiRQsWLlx4KElU9dgi8nN++yhiVbT2UUSir6EigW7BZOUWj1VITgrdReaBnIrHP6TUrRPwuCXbSpRtk73taza9fA8FmT+S1DSVo6+4n8Qjmwd97EDHLy8Wf+0CtQl1/09l+j50NSOBVLmPQqQ66ifFk5tfREHRTycjdeKMpITqX8juX5vB5pnjcfm51G/fkdSB/0d83QYROXa0CMfAA5HyKFFEUDSf1YXrPnrpvgaAwWe2/Vmpa3/HLXsFVtLm6aef5sb778MVFTF8+HAmTZpEYmJipY8d6PiVbReNfRReXsVKzaFEIUD41lYOZanroqIixo4dyz/+8Q8A/vrXv3L33XdjVv6qvCqzLRIaniQKM2sMTAPaARuBgc65H8tpV0hxjSmA75xzv41UjLVN6U7fUC5uE6pS1wV5OVx++eW89tpr1KlTh2effZbhw4dH5NgitZ1XVxR3AO865x4yszt8j/9UTrts51zHSAZW2y7VI7G2sr+1nYPZVnBwL5te/ivZW78ioV4Duo1+kPScE0gP4r9VoPWqNc9CJDCvevb6UlxwEN+//TyKo9br2OZIjqr/0/39OIOj6ifSsc2RHkb1k9xdm/n22ZvI3voVyY2bc97YCTQ/qdyBGSISJl5dUTR3zm33/f490LyCdnXNbAVQADzknHu9oh2a2ShgFEBqamqVA4vmDudwCffaysF0WJe37bzb/sXi5+4gP+sAjVJPYs3SBbRo0SJkxy69XUQqFrZEYWbvAOX9Vd9V+oFvTe6KJnMc7ZzbambHAO+Z2Srn3LflNXTOTQImQfE8imqEXut42elbUSf6Sy+9xAf/vImignxanXY2Xa6+t9JJQkRCI2yJwjl3fkXbzOwHM2vpnNtuZi2BHRXsY6vv3/Vm9j7wK6DcRCFV52Wnb9lOdOcc48eP5667is8njjv3MjoOHENcnMqOiXjFk5nZZvYosLtUZ3Zj59ztZdo0ArKcc7lm1gRYAvR1zq0JtP9onZntj9e3QCJdPbaiCrEUFbDp0X5gRosLR3FU536Hhr9WJbZA7ysS1WP9CfetTtWZkmD5m5ntVWf2Q8AFZvYNcL7vMWbWycye9bX5BbDCzD4DFlDcRxEwSUhsKNuJbkDB5s/Z/K8RxCck0m3UAzTp0r/CORIiEjmedGY753YDPct5fgUw0vf7R0DoelSjnNdnfOE88wxYIRaHc46snZs5KjmBue8upHPnzhXOzA7FsYPdXpl2OnuXmkozs8UzuzJzOcKy2TD9Qeoc25kjW6TyTkYGxxxzjNehiUgpShTimX5NdjL5seEU5Gbx6zYpzHr8AVVQVaoAAA1xSURBVBo3Ds0kPxEJHSUK8cTEiRO5/vrrKSwsJPXMC/nf/+aQlJRUbttw3srRbSKRwJQoJKKKioq48847eeSRRwD4RdoIOlzyhwqThIh4T4lCIiYnJ4ff/e53zJgxg/j4eCZOnMj/Ck72OiwRCaDmrOIiUS03cy89e/ZkxowZpKSkkJ6eztVXX+11WCISBF1RSNgd+GEzHz51K5k7t9CmTRvS09M59dRaM/JZJOYpUUhYffTRR7z7yCjyDu6jY8eOzJs3j1atWnkdlohUgm49SdjMmDGD8847j7yD+2jZoSsLFy5UkhCJQUoUEnLOOR599FEGDhxIbm4ux/66P92vfZiUlBSvQxORKtCtJwmpgoIC/vjHPzJhwgQAHnnkEZandFfNJpEYpisKCZnMzEz69u3LhAkTSEpKYvr06YwdO1ZJQiTG6YpCQmLbtm306dOHlStXctRRRzFnzhy6devmdVgiEgK6opBqW7VqFZ07d2blypUcd9xxZGRk1MokkVdQxJpt+9kRwRUCRSJBiUKqZf78+XTv3p0tW7bQvXt3lixZwnHHHed1WJ4ovVqfSE3iyQp34RaLK9x5rSprKUyePJnRo0dTUFDAwIED+e9//0vdunUPW0fC38pvVd0WrFCt3OdvPxWt1mcGZ7XzthpuOFcuLEsFFmNbNK5wJzHMOcf//d//cfXVV1NQUMCf/vQnXn75ZerWret1aJ4ou1pfnMFR9RPp2OZID6MSCR11Zkul5Obm8vvf/56XXnqJ+Ph4nn76aUaPHv2zNmXPLP1drVR1W7BCtepcoP0cWq3PwAG9OrRgXH/vy5Ro1T0JBSUKCdqePXvo378/CxcupEGDBkyfPp1evXp5HVZU2JWZS7OUJJqlJNExtRE71aEtNYgShQRl/fr1pKWlsXbtWlq1asW8efPo2LFjWI8ZS2fBE4d1OnT2Pq5fB4+jEQkt9VFIQEuXLqVLly6sXbuW0047jaVLl4Y9SYhI9NAVhfg1a9YsrrjiCnJycrjwwguZMWMGDRuGfwRNtImlqxuRUPPkisLMLjez1WZWZGblDsfytbvYzNaa2TozuyOSMdZ2zjkee+wxLr30UnJychg5ciRvvPFGrUwSIrWdV1cUXwADgIkVNTCzeOBp4AJgC7DczOY459ZEJsTaq7CwkJtvvpknn3wSgAcffJA77rgjJms26UpApPo8SRTOuS+BQF88ZwHrnHPrfW1fAfoCShRhdPDgQYYMGcLcuXNJTExkypQpDBkypFr71Je1SGyL5j6K1sDmUo+3AJ09iqVWyN63m3POOYePP/6YRo0aMXv2bHr06OF1WCLisbAlCjN7B2hRzqa7nHOzw3C8UcAogNTU1FDvvsbbt209Hz51G1l7vueYY44hPT2dE0880euwRCQKhC1ROOfOr+YutgJtSz1u43uuouNNAiZBca2nah67Vnnvvfd479FryM/OpHPnzsyZM4dmzZp5HZaIRIlonkexHDjezNqbWSIwGJjjcUw1ztSpU7n44ovJz85kwIABLFiwQElCRH7Gq+Gx/c1sC9AVmGdmb/ueb2Vm6QDOuQLgBuBt4EtgunNutRfx1kTOOe69916GDx9Ofn4+t956KzNmzKBevXpehyYiUcarUU+zgFnlPL8NSCv1OB1Ij2BotUJeXh5/+MMfmDp1KnFxcTzxxBNcf/31XoclIlEqmkc9SRjs3bv30C2m5ORkpk2bRp8+fbwOS0SimBJFLbJx40Z69+7NmjVraNGiBW+88QZnnHGG12GJSJSL5s5sCaEVK1bQpUsX1qxZwymnnEJGRoaShIgERVcUtcDcuXMZPHgwWVlZnHfeecycOZMjj4yO1dc0a1sk+umKooZ76qmn6NevH1lZWYwYMYI333wzapKEiMQGJYoaqrCwkFtuuYU//vGPFBUVcd999zF58mQSExMDv1hEpBTdeqqBsrKyGDp0KLNmzSIhIYH//Oc/DBs2zOuwRCRGKVHUMDt27OCSSy5h2bJlHHHEEcyaNYvf/OY3XoclIjFMiaIG+eqrr0hLS2PDhg0cffTRpKenc/LJJ3sdlojEOPVR1BALFy6kW7dubNiwgU6dOpGRkaEkISIhoURRA7z00ktccMEF/Pjjj/Tt25f333+fFi3Kq/AuIlJ5ShQxzDnHAw88wJVXXkleXh433ngjM2fOpH79+l6HJiI1iPooYlR+fj7XXHMNkydPxsx47LHHGDNmjNdhSZTRhEYJBSWKGLRv3z4uv/xy5s+fT7169XjppZfo16+f12GJSA2lRBFjNm/eTO/evVm1ahXNmjVj7ty5nHXWWV6HJSI1mBJFDFm5ciW9e/dm+/btnHTSSaSnp9O+fXuvwxKRGk6d2TEiPT2dHj16sH37ds455xw++ugjJQkRiQglihgwYcIELrnkEg4ePMjQoUN5++23adSokddhiUgtoUQRxYqKirj99tu59tprKSoq4i9/+QtTp04lKSnJ69BEpBZRH0WUys7OZvjw4cyYMYM6deowadIkrrrqKq/DEpFaSIkiCu3atYu+ffvy0Ucf0bBhQ2bOnMn555/vdVgSgOYsSE2lRBFlvvnmG9LS0li3bh1t27YlPT2dDh06eB2WiNRi6qOIIosXL6Zr166sW7eO008/nYyMDCUJEfGcJ4nCzC43s9VmVmRmnfy022hmq8zsUzNbEckYI23atGn07NmT3bt307t3bz744ANatWrldVgiIp5dUXwBDAAWBtH2N865js65ChNKLHPO8fDDDzN48GByc3O59tpref3112nQoIHXoYmIAB71UTjnvgQwMy8OH1VuvPFGnnrqKQD+9re/ccstt+hzEZGoEu19FA74n5l9bGaj/DU0s1FmtsLMVuzcuTNC4VXfr371K+rWrcuMGTO49dZblSREJOqYcy48OzZ7Byhv9Zy7nHOzfW3eB25zzpXb/2BmrZ1zW82sGTAf+KNzLuDtqk6dOrkVK2KnS2PLli20adPG6zBEpBYzs48rusUftltPzrlqD/x3zm31/bvDzGYBZxFcv0ZMUZIQkWgWtbeezKy+maWU/A5cSHEnuIiIRJBXw2P7m9kWoCswz8ze9j3fyszSfc2aA4vM7DNgGTDPOfeWF/GKiNRmXo16mgXMKuf5bUCa7/f1wC8jHJqIiJQRtbeeREQkOihRiIiIX0oUIiLilxKFiIj4FbYJd14ys53AJj9NmgC7IhROLNLn458+n8D0GfkXjZ/P0c65puVtqJGJIhAzW1FTiwyGgj4f//T5BKbPyL9Y+3x060lERPxSohAREb9qa6KY5HUAUU6fj3/6fALTZ+RfTH0+tbKPQkREgldbryhERCRIShQiIuJXrU0UZvaomX1lZp+b2SwzO9LrmKKJmV1uZqvNrMjMYmYYX7iZ2cVmttbM1pnZHV7HE03MbLKZ7TAzLQdQDjNra2YLzGyN729rjNcxBavWJgqKV8zr4Jw7DfgauNPjeKLNF8AAauBCUVVlZvHA00Av4GRgiJmd7G1UUWUKcLHXQUSxAuBW59zJQBfg+lj5/6fWJgrn3P+ccwW+hxmAlpkrxTn3pXNurddxRJmzgHXOufXOuTzgFaCvxzFFDd8yxXu8jiNaOee2O+c+8f1+APgSaO1tVMGptYmijN8Db3odhES91sDmUo+3ECN/6BJdzKwd8CtgqbeRBMeThYsixczeAVqUs+ku59xsX5u7KL4kfDGSsUWDYD4fEQktM2sAzARucs7t9zqeYNToROGcO9/fdjMbAfQBerpaOKEk0Ocjh9kKtC31uI3vOZGgmFkCxUniRefca17HE6xae+vJzC4Gbgd+65zL8joeiQnLgePNrL2ZJQKDgTkexyQxwswM+A/wpXPuH17HUxm1NlEATwEpwHwz+9TMJngdUDQxs/5mtgXoCswzs7e9jslrvsEPNwBvU9wROd05t9rbqKKHmb0MLAFONLMtZna11zFFme7AMOA833fOp2aW5nVQwVAJDxER8as2X1GIiEgQlChERMQvJQoREfFLiUJERPxSohAREb+UKESqwMz6mZkzs5MCtBthZq2qcZxzzeyNqr5eJBSUKESqZgiwyPevPyOAKicKkWigRCFSSb5aPWcDV1M8O7vk+T+Z2Soz+8zMHjKzy4BOwIu+yVX1zGyjmTXxte9kZu/7fj/LzJaY2Uoz+8jMToz8OxMpX42u9SQSJn2Bt5xzX5vZbjM7A2jme76zcy7LzBo75/aY2Q3Abc65FQDFVRzK9RXQwzlXYGbnAw8Cl4b/rYgEpkQhUnlDgH/6fn/F99iA50rqhjnnKrsuwxHAf83seMABCSGKVaTalChEKsHMGgPnAaeamQPiKf5inxHkLgr46ZZv3VLP3w8scM71961V8H4o4hUJBfVRiFTOZcDzzrmjnXPtnHNtgQ3APuAqM0uGQwkF4ADFxSdLbATO8P1e+tbSEfxUsnxEeEIXqRolCpHKGQLMKvPcTKAlxSXHV5jZp8Btvm1TgAklndnAvcA/zWwFUFhqH48A481sJbrSlyij6rEiIuKXrihERMQvJQoREfFLiUJERPxSohAREb+UKERExC8lChER8UuJQkRE/Pp/2NdYMRqo6UkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "posterior = cv_results.posterior\n",
    "mean = posterior.mean\n",
    "cv_error = ((cv_folds.test_Y.squeeze() - mean.squeeze()) ** 2).mean()\n",
    "print(f\"Cross-validation error: {cv_error : 4.2}\")\n",
    "\n",
    "# get lower and upper confidence bounds\n",
    "lower, upper = posterior.mvn.confidence_region()\n",
    "\n",
    "# scatterplot of predicted versus test\n",
    "_, axes = plt.subplots(1, 1, figsize=(6, 4))\n",
    "plt.plot([-1.5, 1.5], [-1.5, 1.5], 'k', label=\"true objective\", linewidth=2)\n",
    "\n",
    "axes.set_xlabel(\"Actual\")\n",
    "axes.set_ylabel(\"Predicted\")\n",
    "\n",
    "axes.errorbar(\n",
    "    x=cv_folds.test_Y.numpy().flatten(), \n",
    "    y=mean.numpy().flatten(), \n",
    "    xerr=1.96*sigma,\n",
    "    yerr=((upper-lower)/2).numpy().flatten(),\n",
    "    fmt='*'\n",
    ");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can visualize the fitted models. To do this, we again take advantage of batch-mode evaluation to obtain predictions, including lower and upper confidence regions, from each of the 20 models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = cv_results.model\n",
    "with torch.no_grad():\n",
    "    # evaluate the models at a series of points for plotting \n",
    "    plot_x = torch.linspace(0, 1, 101).view(1, -1, 1).repeat(cv_folds.train_X.shape[0], 1, 1)\n",
    "    posterior = model.posterior(plot_x)\n",
    "    mean = posterior.mean\n",
    "    \n",
    "    # get lower and upper confidence bounds\n",
    "    lower, upper = posterior.mvn.confidence_region()\n",
    "    plot_x.squeeze_()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code snippet below plots the result for the 12th CV fold (by setting `num = 12`), but note that we have computed the results for all folds above (other plots can be obtained by iterating `num` from 1 to 20)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dd3xUVfr48c+ZmcxMJr33hN47ERErNhApdkVdy+oiurr+1F236K5+ca37XfdrV3ZtKItiQVlAReyKiAGk95bee5l+fn8kWREDCWRm7p2Z8369eJlkrnPPTXnmznPOeR4hpURRFEUJfQatB6AoiqIEhgr4iqIoYUIFfEVRlDChAr6iKEqYUAFfURQlTJi0HsCRJCcnyz59+mg9DEVRlKCybt26aillSleP6Tbg9+nTh4KCAq2HoSiKElSEEAeP9JhK6SiKooQJFfAVRVHChAr4iqIoYUIFfEVRlDChAr6iKEqYUAFfURQlTKiAryiKEiZUwFcURQkTut14pfhHk91FRaODyiY7LQ4PdpcHp9uL0SCwRhiwRBhJsJlJjbGQHG3BbFL3BIoSKlTADwOVTXZ2lTezq6KJhjZXj/8/ISAzLpJ+KVH0T4kmIcrsx1EqiuJvKuCHKCkluyqa+f5ALVVNjuN8Diipb6Okvo2vdleTGW9ldE48A1NjMBqEj0esKIq/qYAfgnaUN7Jmbw11rT2/m++J0no7pfXlfGmpYlxuAqNz4okwqpSPogQLn/y1CiFeEkJUCiG2HOHxM4QQDUKIHzr+/cUX51V+qq7Fydvrivlgc7nPg/2hWhwevtpdzcvf7GdDYR0er+qLrCjBwFd3+K8ATwMLjnLMV1LK6T46n3IIKSVr99eydn8t7gAG3xaHh893VrGpuIEzBqeQlxQVsHMrinLsfBLwpZRfCiH6+OK5lGPT4nDz4ZZyCmtbNRtDbYuTd9eXMDAtmjMGpxJtUZlCRdGjQCZgTxJCbBRCfCCEGN7VAUKIOUKIAiFEQVVVVQCHFpyKaltZ+N1BTYP9oXZXNPPatwfZXtao9VAURelCoAL+eiBPSjkaeAp4r6uDpJTzpZT5Usr8lJQuG7YoHbaWNrBkQwktDo/WQ/kJu8vDh1vKWbqxlFanW+vhKIpyiIAEfCllo5SyuePjFUCEECI5EOcORav3VrNya8URJ0sbayp5+q6raazV7l3S3spmFq4ppEgn7z4URQlQwBdCpAshRMfHEzrOWxOIc4cSKSUrt5bz3b7aox63cuGz7N9SwMrXnwnQyLrW7HDzzvpiVu+txqtW8iiK5oSUvf9DFEIsAs4AkoEK4D4gAkBK+bwQ4lbgZsANtAF3SilXH+058/Pzpepp+yOvV7JyWznby5qOeMzd00fhdv58k5XJbOGxZZv8Obxu5SbamDYyg0izUdNxKEqoE0Ksk1Lmd/mYLwK+P6iA/yOvV/Lh1nJ2lh852EN7Kmfp/EfZvHoVLoedCIuVkSefw8w5vyc2Ufs5kdjICGaMyiA11qr1UBQlZB0t4KttkjonZc+CPUBsUipWWzRupwOT2YLb6cBqi9ZFsAdobHOxuKCIHeVqFY+iaEEtmNa5T7ZX9ijYd2qqr2HS9NlMnHY5a1a8qenEbVdcHskHm8upbXZyUv8kOqZ2FEUJAJXS0bFv9lSzdv/RJ2iD2aC0GM4dnqbq8SiKD6mUThBad7AupIM9wK6KJt5ZV6zW6ytKgKiAr0O7K5r4are+UjH+UtZg583vi6hrcWo9FEUJeSrg60x5g52Ptpaj00ybX9S3unizoIiyhjath6IoIU0FfB1paHOxdGMJLk8YRfsObU4P76wrZn91i9ZDUZSQpQK+TjjdXpb+oL/aOIHk8kiW/lDKtlK1bFNR/EEFfB2QUvLR1nKqm1Ue2yvbdxSvOxjaE9aKogUV8HVgzb5a9lQ2az0M3ZASvtxVzTd7qrUeiqKEFBXwNbanspnv9qs6cl1Zu7+WT3dUoNe9IooSbFTA11Bdi1OXK3L0UF6508aiBj7cUq6qbSqKD6iArxGn28uyTaU43V6th/Izeimv3GlHeRPLNpfh9ujve6UowUSVVtDIB5vL2HEMNXICQc/llQHykmzMGJ2pSjEoylGo0go680NRve6CPcC9r65i3OTpRFjayxdHWKyMO3MG9y74ROORtTtY08qS9SU43OG7dFVRekMF/AArb7Dz5S7tc+Nd0Xt5ZYCS+jaWrC/B7lJBX1GOlQr4AWR3eVi+ueyIvWj1oLO88u1PLGbS9Nk01elvaWRZg5131hfT5lRBX1GOhcrhB9DSjaXsVevtfSY52sxF47KJsqi2DorSSeXwdWB9YZ0K9j5W3ezk7XXFNDtUeWVF6QkV8AOgvMHO17v1lxoJBbUtTt4qKKLR7tJ6KIqieyrg+5nd5WGFzvP2wa6+1cVbBcU0tKmgryhHo5KffvbJ9krdBCKnXVC828qB7VaKd1tpqDHS2mikpdGIySSxRnmJjPaSlOEka4CDrP4OcgbZsUTq/8Wqsc3FWwVFXDI+m3ibWevhKIouqYDvRxuL6tlVoe16e7cLdnwfxbpPYtm6Jgq3q/1NXWKai4R0F+l5TiJjPHg9grYWA61NRnatj6JgVRwAJrOXQWNbGTGpmZEnNxMVq9/drk12N2+vK+bicdkkRKmgryiHUwHfTyqbtF1vb28VfP1+PF+8m0BLg4noODcTpzUweHwruYPtxCQcfUljU52R4t0WdhREseXbaLZ9F827T3sZc3oTJ89sIG+IPUBXcmya7G7eWlfExeOySYq2aD0cRdEVtSzTD5xuL4vWFlKrQZ9Wl1Pw5ZJ4Pn8rkZZGI0NOaOGUWfUMHteC8Thf3qWEkr0W1qyIY90nsTjaDPQd3sbUa6sZOEafbQltZiMXjcsmJUYFfSW8HG1Zpgr4fvDhljK2lwU+lbNvi5XF/0ijssjCkBNamPKLGp/fidtbDKxdGcunixNprDExYHQr5/+ymryh+rvjjzQbuWhsFqmxVq2HoigBowJ+AG0paeDjbRUBPaejTfCff6awelk8CWkuLr29giH5rX49p9Mh+HZ5HJ+8kUhzvYkTpzZw/i+riY7X1+5Xa4SRC8dmkR6ngr4SHlTAD5CaZgeL1hYGtAl5ZXEEr8zLpOKgmVMvqOe866oDuqrG3ir4eGESX7ybgMXmZfovq5k4rQEhAjaEbplNBi4Ym0VWfKTWQ1EUv/P7TlshxEtCiEohxJYjPC6EEE8KIfYIITYJIcb54rx64nR7Wb65LKDBfuNX0fzj1lyaak3MebCEC26uCvgSSqtNMuNX1fz2+YNk9nPw1hNpvPDHLOoq9bMewOn28t6GEopq/fuuR1H0zlcbr14Bph7l8fOAgR3/5gDP+ei8uvHpjgpqAtSEXEr4eGEirz6QSVqukzufPchgP6dwupOe5+SWx4q55DcVHNgWyd/m5LF2Zaxuunk53V7e/6GEA9UtWg9FUY6qqsnht2Y/Pgn4UsovgdqjHDILWCDbrQHihRAZvji3HmwpaQjYJK3XA+88lcoHryYz/qxGbv3fYhJS9VFLRgiYNL2B371wgKwBDt7433QWPpKOvVUf+R2XR7J0Y6lqGK/o2u5K/8WSQJVWyAKKDvm8uONrPyGEmCOEKBBCFFRV6bNm/OEqG+18tqMyIOdyOQWvPJDJ6mXxnHl5LVfeXY7JrJNb6EMkZbi5+dFizru2mg1fxPD4LXkU79bH8kiPV7J8U5nmG+IU5Uj8WWRRV7V0pJTzpZT5Usr8lBT9NN04ErvLw382leEOQJ0cl1Pw0v2ZbP02igtvqWT6DdW6mhg9nMEI51xVy6//VozLKXjy/+Xw3YexWg8LAK+UfLC5nC0lDVoPRVF+orbFSbUfU8OBCvglQM4hn2d3fC1oSSlZsbmMxgDUyekM9rvW2bjsjgpOvaDe7+f0lX4j27jruYP0HdHGm4+n8/aTqbh1UFrIKyWrtlewobBO66Eoyn/5O90YqIC/FLimY7XORKBBSlkWoHP7xeq9NRys8f9EqcspePn+THYWRHHZHRWcOLXR7+f0teg4L3MeKuHMy2tZvSyeZ3+XQ1OdUethISV8vrOKtfuPNv2kKIHjz/w9+G5Z5iLgW2CwEKJYCHGDEGKuEGJuxyErgH3AHuCfwC2+OK9WdlU08f0B/wcJrwdefySdHQVRXHZHeVAG+05GI0y/oZpr7i2lZK+F/7stl5K9+ihw9s2ear7aHRxzRkroamh1Udno8Os5fLJYWko5u5vHJfBrX5xLaxWNdlZuLff7ckMp4e2nUtn8dQwX3FzJxPOCN9gfasxpzSRnFPHifZk8dUcuV/+xjBEnab9UsuBAHQ6Xl7OGpiL0PDmihKw9Vf5fSKCrSVu9a7K7WPpDqd83VzXWVPLAL75kzYp4zrqihtMuDJ6cfU9kD3Tw/54qJC3Xycv3Z/LlknithwTA5pIGVmwuV81qFE3srvD/cmEV8HvI6faydGNpQPqnLnhoB/WVN5Ka8xnTrq/x+/m0EJfk4df/W8Twk1p477lU3n8+Ba8OSu3vqmji/R9KcLp1MBglbDTaXZQ3+r8AoQr4PeDxSpZtKvV7fu3u6aO489xfs2/zdcByKovO4a4pg7l7+ii/ntfXGmsqefquq2msPXpe3GyVXPfnUk69oI4v3k1gwV8zcDm1T6ccrGnlnfXFtDn1VQhOCV27K5oCsitdBfxueL2SD7aUBWRFzk0PfYPRtATEduAKIiwRjDtzBvcu+MTv5/allQufZf+WAla+/ky3xxqMcOEtVcy6qZJNX8fwwh+zaGvW/teyvMHOW+tUc3QlMHaUB2YjoH4qXOnUJzsqA5Jba643suh/R2IwtuJxT8dkduF2OrHaoolN1P8mNGh/h+J2/vguaPWyRaxetgiT2cJjyzYd9f89/eJ6YhM9/Ptv6Tx9Vw5zHiomLknbO+yaZieLvy/igrFZJKvuWYqf1LY4/Z496KT9rZROeb2Sj7dVBGQ3pscNr/41g8ZaE7mD/8zJM07h9icWM2n6bJrqqv1+fl+599VVjJs8nQhLe+35CIv1mN6hjJ3cxK/+WkJteQRP/r9cqkoi/DncHmmyu1lcUERxnaq0qfjHzgDd3YO6w++S2+Plgy3lASuy9f4LKezdZOOq35cx/qzb/vv1i2+7LyDn95XYpFSstmjcTgcmswW303HM71AGjWvllr8VMf+eLJ66I4ebHi4hq39g7n6OxOHysmR9CVNHpDMwLUbTsSihZ2d54JZcqzv8w9hdHt77IXAVFdd+FMvX7ydwxiW1jD8r+At6NdXXMGn67F69Q8kZ5ODWvxdhipA889ts9m3RvluV2ytZvrlMlWJQfKqy0U5da+DmiVTHq0OUNbTxweZyGgJQHwfg4A4rT9+VTf8RbfzqoRKM2lcb0IXGmkoWPHQnM296hoWPjKa+ysT195X6vW1jT43PS+DUgclqg5bSa1/uqmLdwZ/fRNx25gBMxuO7H/d7x6tgJ6Vk7f5aFn9fHLBg31xv5NV5GcQlevjFPWUq2B+ic5XP2g//wa1/LyI128mL92Wy+ZsorYcGwLqDdXywpdxvTSqU8CClDHiZ7rDO4Usp2VPZzJr9tVQ3BS5P3Fkjp7nByG/+r4ioWBU44MirfIwRaWQP2M+rD2Qy+3flukh97SxvotnhZuboTKwR6tVaOXbFdW002QPbvCgk7/CLalv5fGclxXWtHJ6yklJS3exgY1E9r685yLJNZQEN9gAfvZ7ErvVRXHRrJdkDtZ2Q1JMjrfL582vvcNPDxfQd2ca/H0tnzQf6qKtfUtfGm98X0RDAHKwSOraWBr4+Vkje4be5PGworGdDYT3WCCNRFiMmgwGjAWpbXNhd2q3v3rY2io8XJjFhSkPIFETzlaOv8pH86q8lvPI/mSz+Rzoel+Dkmdo3MKltcfJmQSEzR2eRHqf95LISHBxuD3v8XAq5KyF5h38ou8tDTbOTikY7pfV2TYN9XaWJfz+aTmY/OxfdGpi2iMHmaKt8zBbJL+8vZfjEZt55Oo0v3tFH0bUWh4e31xWxt0r1ylV6ZndFs9+LMHYlJO/w9cjtggUPZuDxwLV/LsNs0efqKK1df9/T//24q30IJrPk2j+XsvCRDN5/IRWPR3DmZdovlXR5JP/ZWMoZg1MZk6OPFyJFv7ZpkM6BMLjD14vlL6ZwcHskl99ZQUqWyvn2hikCrv5TGWMnN7LsXymsWpSg9ZCA9h4Gn+2o5ItdVT+bO1KUTvWtTkrq2zQ5twr4AbD5myi+eDeBk2fWM+Y09bbfF4xGuPLucsad2ciKl1NY+Xrifx/rabVOf1l/sI7lm8vUsk2lS1rd3YMK+H5XW27ijb+nkz3Qzqw5qo2eLxmNcOXvysk/u4EPFyTzUUfQP5Zqnf6yu6KZd9eXaDpnpOiPlJJtZdoFfJXD9yO3CxY8lIH0wjX3lGEyq7f5vmYwwhV3VSAEfLQgmY8WPAksAo6tWqc/lNS3L9u8YGwWcZHaF4JTtHewpjXga+8PpQK+H614OZnCHZFcc28pyZn6ydtbI4xkxFmJsZqINBuxmU14pcTp9uJ0e6lvc1HV5KAxQLuOe8tghMvvrMDpsLPxy3kYjCa8nvuIsFgZefI5zJzze83GVtvi5M3vC7lgTBapsWrZZrjbWKxtu1IV8P1k23dRfP52IpOm6yNvnx5nZUh6DLmJNhKjzD2qA2N3eSiua2VPZQsHalp03QHKYIRf/LGB0n07qCr+CwajF7dzni76CbQ4PLy1rpgZozLJTbJpOhZFO412Fwequ68H1VhTyZmTb2Tx4sWkp6f7dAwq4PtBfZWJRX9rX28/a652eXuTQTAiO45RWXEkHUcDD2uEkQGpMQxIjcHrleyrbmFTcT2Fta0Bacd2rAxGSMt9GDBQVXw/eUOG0FT3ktbDAtp7Ir/3QwnnDk9jSLo+dgorgbWluAFvD/5wVi58ljXffMO8efN49tlnfTqGkKyWuauiieWbynw8op7xeOC532VTvMfKnc8eJDU78GkRgxAMz4zlxH6JxFh9nzuua3Hy/YFatpc19egXONC8Hvj3Y+ms/yyW6TdW6WKdfich4LRBKYzL1cdSUiUwPF7Ji1/vo8Vx5HfJh9eS6mS1Wmlr6/kyTlUtM4BWvpbEvi02Lr29QpNgnxFn5eqJuZw9LM0vwR4gIcrMucPTueakPIakx6C3KsEGI8y+u5yxZ7Sv0//iXf1shJISvthZxeo9wdPJTOm9vVXNRw328PNaUjabjauuuor9+/f7bBwqpeNDuzZEsmpRIhOmNAS8omOEUXBS/yTG5SYErE57QpSZ80ZmMDY3gU93VFLRaA/IeXvCaIQrf1+O1yN4//lUjCbJKTqovdPpu/21tLk8nDkkVdXVDwMbi7qfrD20lpTVasVutxMbG+vTPL66w/eRxlojCx/JIDXHyYW3BLZOToItgism5DI+L1GT4JEeZ2X2hBzOGpqKJUI/v1JGI1z9xzJGnNTMu0+n8e2KOK2H9BObihv4cEs5Xq/+0mKK79Q0Oyiu61lKprOW1NffrGbu3LmUl5f7dCzqDt8HvB5Y+Gg69lYDNz9ajCUycH/A/VKimDI8XfOa7EIIRmXH0y8lmlXbKthf3aLpeDoZTe17IF6el8HbT6RiMklOOFc/VUp3lDfh8kqmjUg/7g5Hir6tL+z5UszOWlKjRw/gmWd8v3HQJ79hQoipQoidQog9Qog/dPH4dUKIKiHEDx3/bvTFefXikzcS2b0higtvqSS9jzNg5z2xb6LuGnBEW0xcMDaLc4alYTbpI4CZzJLr/lLGwDGtvPF4Ghs+j9Z6SD+xt7KZpRtLcalSDCGnxeFmh4Y7aw/X679IIYQReAY4DxgGzBZCDOvi0DellGM6/v2rt+f1p2OpxbJ3UyQfvpbEuDMbOXFqYH6wQsDkIalMGqDfvqojsuK46sRc0nSy2SjCLLn+/lL6Dmtj4SMZummX2OlgTSvvbSjB6VZBP5RsLKrHraOUnS9uwSYAe6SU+6SUTuANYJYPnlczPa3F0lRn5LWHM0jOcHHJbyoCslrFaBCcNyIjKErwxtvMXH5CDmNz9TFWS6Tkxr+WkD3IzoIHM9m+Vl+boIrr2njvBxX0Q4XL42VTiX4WCoBvAn4WUHTI58UdXzvcxUKITUKIt4UQOV09kRBijhCiQAhRUFUV+A1Ld08fxZ3nDmb1skVIKVm9bBF3njuYu6eP+tmxXm/7Wu/WRgPX3FuK1eb/V3GTQTBjdCaD02P8fi5fMRoEZwxOZcboDF2keKw2yU0PlZDex8HL8zLZ/UOk1kP6iZK6NpZsKMbh1u+uZqVntpU26m53eqD+Av8D9JFSjgI+Bl7t6iAp5XwpZb6UMj8lJfDb4Y/UU/XeBZ/87NhP3khk57ooLryliqz+/s/bGw2C80dl0DdZX6mInhqQGsPsCbkkRpm1HgqR0V5ueriY5AwXL/4li/1b9ZF26lRab1fpnSAnpWRDoX42/HXyRcAvAQ69Y8/u+Np/SSlrpJSdW8j+BYz3wXl97ug9VX+0Z1MkHy5IYtzkRiZO8/9bNoMQTBuZTr8UfU02HqvEKDNXTMihX4r2L1rRcV7mPlJMbKKbf96TRfHuH0tPaF1PHzqCvkrvBK29Vc3U6bC5vS8C/vfAQCFEXyGEGbgCWHroAUKIjEM+nQls98F5/eJoPVWhfb39aw9lkJzp4pLb/Z+3FwKmjkhnQGrwpHGOxmIyMnN0Jif0Sez+YD+LTfJw82PFRMZ4eeGP2ZQfbH/3oYd6+tCe3nn/hxK1eifISClZs69W62F0ySe1dIQQ04D/A4zAS1LKB4UQ84ACKeVSIcTDtAd6N1AL3Cyl3HG059RjLR2vB57/QzYHd1i5/clCMvv6P5Vz+uDQrbuyrbSRT7ZXaL6KoaokgqfvyqGpthI4Fdj3k8e1qqffKTfRxqwxmWqdfpDYU9nMfzaW9uo5bjtzwHH/vP1eS0dKuUJKOUhK2V9K+WDH1/4ipVza8fEfpZTDpZSjpZSTuwv2evXhgiT2bLRx8W2VAQn2+X0SQjbYAwzLjOXCcVma7yNIyXJx8yPFREanYLauxmQeABx9DieQCmtbWb65DI+OlvcpR/bd/hqth3BE6pahh7avtbFqURITpjQwIQA7NYdmxHDKgGS/n0dr2Qk2Lj8hR/OOUOl9nMx9pBSPOwa3cxnGiOwjzuFoYV9VCx9sKVNlGHRub1UzlY0/r3ipFyrg90BtuYnXH80go5+Di271f52crIRIzhmWrttNVb7WOZmbHqftapmcQQ7yht6DwdCHxLQtTJhyw8/mcLS0u6KZldvK0WtJcwW+02nuvpMK+N1wOQWvzMtEeuG6P5ditvj3jy0uMoIZozIxGsIj2HeymU1cPC6bPsnaboa69e83c+Nfq6gtj6Wi8EmuvFvbidvDbS9r4rOdgS3Op/TM3qpmXVWM7YoK+N149+lUivdYufLuclKy/LvMymwyMGtMJpFm/dTGCSSzycCs0VkMzdC2I9SQ/Fau/mMZB3dYefn+TNxOfb34bixq4Kvd2i0ZVX7O65V8EwQ9DlTAP4o1H8Ty3YdxnD27hhEn+bf6oxBw/siM42pFGEoMBsGU4WmMz9N2snr0qc1cfmcFuzZE8drD6Xj0tWGSggN1fLdPv5OD4WZbWSM1zYErnHi8VMA/goM7rLzzdCqDxrYw9Rr//2GdPCCZPkG6i9bXhBCcNiiFUwZqO2k94dxGLri5ks3fxPDm39Pw6mw5/Oq9NbrczRluXB4va3z44uv1482FCvhdaKoz8sq8DGITPVz9pzIMfs6wDEqL0cVGJL05oU8iZw9N07SF4mkX1jPlmmoKVsXx3nMpumve/sWuKraW6qtAV7jZUFhPk93ts+db8lwq11wj/PK7pgL+YTxuePWvGbQ2Gfnl/aVEx/n3ti45xsI5w9L8eo5gNjI7jvNGZGg6iX3uVbWcfnEtX7+fwIcLkjQbR1ekhFXbKtldEdiWmkq7NqeH7w/4bmVO6X4zq5fFkZCAX250VMA/zPsvpLBvs43L7qggq//xraftaS0WS4SBmaMydVFFUs8Gp8dw/qgMTBoFfSFg5pxqTpzawMcLk/j8bX2Ue+7klZIPtpTrpstYOFm9t9pn9Y6khPefT8Ea5eUvf/HPW0kVaQ6x5oNYvn4/gdMvrmX8mcd/x9STWixCwNTh6cTZtN1wFCz6p0Qza0yWZi+OQsClt1cw+tQmls5PZc0H2q4kOpzHK1m+qZSi2lathxI2KhrtbPZhvfut30axe0MUU6+pIclPbyRVT9sO+zZH8s5TaQzOb2H6jce3vOru6aNwO398V7B62SJWL1vUZS2WE/okBn31y0DLTbJxwdgszUoHG4xw1e/LsbcZeOuJNCKjvIw+rTng4zgSl0eydGMpF4/L1nwTW6iTUvLZjkqf5dndTsH781NIy3MwaXo94J8FC+oOH6itMPHyvAwS011c86cyjMc5SdvTevq5iTYm9ddXLjhYZMVHcuHYLCwR2vzqmsyS6/5cSp+hdl5/JIMdBfrqmuV0e1myoYTKJn1vAAp2W0sbKWvw3ff4y/fiqSk1c8FNVccdf3oi7AO+vVXw4n1ZeNyCG/6nhMjo479z7Ek9/RirifNGhk/ZBH/IjI/korHZmhVds0RKbnyghPQ8By//T6buGqjYXR7eXV9CdbN+a7oEM7vL49NNVk11Rj7+dyLDTmxmcL5/U3JhHfC9Hnj94QwqDpi55p4yUnN6v5P2aPX0DUJw3sgMbGaVSeut9DgrF2tYaTMy2such0qIT3Hzz3uzKNmrrw1zbU4P764vpq5F/5uBgs1Xu6tp9WHrwg9eScLlMDDzJv/vnvZJPXx/CEQ9/CXPpfDVkgQuvq2Ck2f4fy3zKQOT1Xp7H6tstPPO+hLsLm22wtZVmnjqjhzcLsGtjxeRmq2vLkcxVhOXjM8m3qZ9a8lQUFjTyjvri332fCV7zTx+Sx6nXlDPBTf/GPB1XQ8/GH3znzi+WpLAaRfWBSTY90uJIl/jcgGhKDVW2zv9hFQ3cx8pBgnP/z6bukqTLlokdmqyu3l7XTH1repOv7ecbi8fb6/w2ehpW+wAACAASURBVPO1L8NMJTLay7lXB6ZMRlgG/K1ronj3mVSGndjMzDn+/6OMsZqYMlzl7f2lM+hrNZGbmuNizsMl2FsMPP+HbJa99LouWiR2UkHfN77ZU01jm+/ewW39Noo9G21MuaYGW0xgVp2FXcA/uMPKaw9mkD3AwS8CUDahvQF5huZdnUJdaqyVi8Zma7ZOP3uAA5fzbKqK3RR8fBNSxrJ62SLuPHcwd08fpcmYDtUZ9GtVTv+4FNe1srG43mfP53bB0vkppOU6mHS+7563O2EV8KtKIvjXnzOJSXRz4wMlWCL9P39x8oAkMuMj/X4epX0i94Kx2m3O+vNrDzBg1CPAcGA5JnOiLlokdmoP+kVq9c4xsrs8fLil3Ke1bb5ZGk91qZmZc6owBnANR9gE/KY6I/PvyQIp+NWDJcQk+H+Sr19KlOZlfsNNVnwkM0dnEmEMfPosNimV1JzdwFXARNzORZgt8bpokdipxeHh7XXFVOq8UYeefLK90qfF0VoaDaxcmMTg/BaGTgjszuiwCPhtzQZe+GMWTTUmbphXEpCVFDFWE+eGUZtCPclJtHG+Rl3DmuprOHmGmSm/2AScy7bv5uiuln6b08Nb64pVGYYe2FLSwC4fF6b76LUkHK0GZgVg/vBwIR/wnXbBv/6SSUWhhevuK6XPMP/f2XSutw/XzlV60Dc5ivNGpGMI8Avu9fc9zcW33ceUX0Rxwc2VNNaewpt/T9ddLX2n28t7G0rYU+n7KptlZWWcfvrplJeX+/y5A6muxckXu3wblCsKzaz+TzwTpzWQ3ifw8ykhHfDdLljwYAYHtkZy5d1lDPHzLrZOE/slkqXy9pobmBbDOcO0q6d/2oX1nHddNQWrYnn36VTd1dJ3eyXLN5Wzudi3y5IfeOABvv76a+bNm+fT5w0kp9vLsk2lPq/ZtHR+MuZIb0CaKnUlZLd8ejp20W77LppLflPB2DMCU+QqN9HGhL5qc5VeDMuMxe728MVObdbEnz27Fnurgc8WJ2KJ9DL9xmpNG7oczislq7ZX0Gh3Mal/Uq9SkJGRkdjtP76Dfu6553juueewWq20tbX5YrgBs3JbOdU+blm4o8DG9rXRTL+xiuh4bfJ8IXmH7/HAvx9NZ9PXMcyaW8mk6YHpCBRlMTJ1hMrb68243ARO7KfNi7AQMP2Gak6eUc9nbyXy8UJ93gys3V/LB1vKcXuO/4523759XHnlldhs7QXlbDYbV111Ffv37/fVMANi7f5adlf49gbR42nvtZGU6eS0CwK3DPNwIXeH7/XCPXdY2fB5BOffUMXpFwXmmysETBmeTpQl5L6lIWFS/2QcLi8/FAX+j00IuPDXlTjaBB8uSMZs9XLGJdr90R/JzvImmuwupo/KPK7f44yMDGJjY7Hb7VitVux2O7GxsaSnp/thtP5xoLqF1Xt9Vxit05rlcVQctHD9fSWYzNrl9kLuDn/PHli5wsSUa6o56/LANXgen5dAXpJqQq5nZwxOYVBajCbnNhjg8rt+bKCyelmcJuPoTmm9nUVrCyk/ztK/FRUVzJ07lzVr1jB37tygmritbLSzfHOZz+daWpsMfLggmQGjWxkxSduuZD65HRVCTAWeAIzAv6SUjxz2uAVYAIwHaoDLpZQHfHHuww0aBMu/aGFdte/6THYnPc7KpP7+aVig+I4QginD02h1uimuC3xO2WiEq/5QhtslePvJNCLMkhPObQz4OLrTZHfzVkERk4ekMiLr2F6Y3n333f9+/Mwz+igt0RMNbS7e+8E/jXU+XphEa5OBWXMrNZ+/6fUdvhDCCDwDnAcMA2YLIYYddtgNQJ2UcgDwD+DR3p73aDKyZMC+sWaTgWkaN9lWes5kNDBzTCYpMdqUMzZFwDX3ljFoXAtvPJ7Ghs+0ecfRHbdX8vG2Cj7aWo6rF3n9YNDm9PDehhJaHL6fSK0siuCr9+OZMLWRrP7al7XwRUpnArBHSrlPSukE3gBmHXbMLODVjo/fBs4SITKzedbQVNWXNshYTEYuGJtFjFWb+ZYIs+SX95fSb0QbCx9NZ9PX+m11ua20kUVrC0O2HIPd5eG9H0r8VmNo6fwUzBYv067z/bzA8fBFwM8Cig75vLjja10eI6V0Aw2A33r8VVaUB6Q87dCMWIak66uZtdIz0RaTpq0SzVbJDfNKyB1iZ8GDGWz5Vr/zPzXNTt5YW8iGwjr02j/jeNhdHpZsKDnu+YrubF9rY9t30Zx7dW1ASrn0hK4mbYUQc4QQBUKIgqqq4w/Wzz7+qN/L08bbIpg8RD81UpRjlxRtYYZGJRgArDbJrx4sIXuAnVcfyGTbWv0GfZdH8vnOKt5dX0KTXV9NXo5HZxtIfwV7j7u91n1KlpNTZgVu8Uh3fBHwS4CcQz7P7vhal8cIIUxAHO2Ttz8hpZwvpcyXUuanpBx7MI2MjEQIwaJXX0RK6bfytAYhOG9EBhaTKp0Q7HISbZw7PE2z80dGebnp4RIy+jh45X/01xT9cIW1rby25iCbiuuD9m6/2dFeKrrCjwXkvl4aT2WxmVlzqzDpKOPri4D/PTBQCNFXCGEGrgCWHnbMUuDajo8vAT6Vfvht6dz4YY1sL2sQYbH6pTztif0SSY/TV+Nq5fgNSY9lYj+/ZRi7FRnt5aZHiknLc/LSfZm6D/oOl5dPtlfyVhDW169qcvDG2kKqmvw3J9FUZ+Sj1zqrYWq7DPNwvQ74HTn5W4GPgO3AYinlViHEPCHEzI7DXgSShBB7gDuBP/T2vF3p3PjhsDswmS24nQ6stmiflqfNjLcyQfWlDTkn9U9iaIZ2K2aiYr3MfaSYtNz2oL9T50EfoKSujdfXHOSr3VU43PrIUR/NgeoWFhcU+bTUcVeWv5SM027gAh0swzycT5YpSClXACsO+9pfDvnYDlzqi3N1p6KigtnX/pKME2ewZsWbPp24NZsMTBmejkEtwQxJ5wxLp9HupkSDNfrQEfQfLeb532fz4n2ZXH9/KUNP0HcJY49XUnCgju1ljZw8IJlhGbG6Ky3i9UrW7Kth7YFavxewO7jDytqP4ph8aS1pufqb6xB6zcPl5+fLgoKC4/p/d1U0sXxTmY9HBOcMSzvmjShKcGlzeli0tpAGH/YuPVYtje29ccsPmrn+vjKG6SwtcDTJMRZO6pfEgFR9LDVttLv4cHM5JfX+fxH3euGJ3+TSUG3iDy/tx2o7/th625kDMBmPLwEjhFgnpczv6jFdrdLRs34pUSrYh4FIs5GZYzI1a5MIP6Z3Mvo4efn+TF0v2TxcdZOD/2ws5Y21heyratZsYtfrlfxQVM/raw4GJNgDrP0olqJdVmb8qqpXwd6fVMDvAWuEkbOGareSQwms5GgL541I1zT/GhXr5eZHi8nqb+eVeZn88KU+7ph7qqzBzvs/lPL6moNsK23E4w1cACxraGPR94V8tqMShyswu4RbGg0sfymZviNaGXem75vK+IoK+D1w+qAUolUVzLDSLyWaUwdqWx+pffVOCXlD7Lz2UAYFq/RZhuFoqpudfLS1nH99tY+vdldR3+q/VT1Fta0s2VDMG2uLqGwM7M7g5S8m09Zk5OJb9TdReygV8LvRLyWKYZlqN204Gp+XyNAMbX/2kVFe5jxUzIBRbSz6WzrfLg/OtGKr00PBgTpeWX2AN78vZN3BWp/Mk7Q63WwqrueNtYW8va6YA9WBn+Q+sM3Kmg/iOfXCOjL7OWmsqQzITv/joW5bj0KlcpSzh6ZS1+r0247MnrBESm54oIRXH8jgrSfSsLcamHypfnZvHgsp20swl9bb+XJXNYlRZjLjI8mMt5IaYyUuMuKo8yd2l4eKRjvlDXaK69oormvDq+HCE48H3noyjbhkF1N+0b6XdOXCZ/+70/+S39yv2di6ogL+UZw6MFmlcsKcyWhgxuhMFn1XSLPDv+u3j8ZskVx/Xyn/fiyD//wzhbZmA+ddV6Pr9EFP1LY4qW1xsqXkx650URYjURYTRiEwGgQSaHW4aXV5ApaT76mvlsRTts/CtX8u5S+XjcTt/DGVtHrZIlYvW4TJbOGxZZs0HOWPVDQ7gqyESIarVI5Ce6G16aMzeLugGHcAJx8PZ4qAq/9QhtXmYdWiJNqajVx4SyWGEKvw0eLw+KVUsa/VVZr46LVkhk5oZtQpzfQdtoql8x9l8+pVuBx2IixWRp58DjPn/F7rof6XyuF3wWQQnD00TXcbSBTtZMRFMnlIqtbDwGCES/9fJZMvq+Wb/8Tz2sMZuJ3q9zTQpIS3n0xFeuGiX7dP1MYmpWK1ReN2+m+nf2+pO/wu5PdJJDHKrPUwFJ0ZkRVHZZOdjUUN3R/sR0LAjBuriYn3sHR+Cq2NRq6/rxRrlL7SHaFsw+cxbF8bzay5lSRl/Jjqa6qvYdL02UycdrnPd/r7ggr4h0mMMjOhr6qVo3Tt9EGpVDc5A7aZ52jOuKSO6Hg3b/w9nWd+m82v/lpCbJL+UyHBrrnBwJJnU8gd3Maps37ajP76+57+78cX33ZfoIfWLZXSOczpg1JUu0LliIwGwfmjMnQzmZ9/dhM3zCuhqsTME7fnUn5AvTP1t6UvpNLWbOSyOyuCbv5EBfxD9EuJok9y8GxjV7QR1TGJq5cbg6EntHLr34twuwVP3pHD7h8itR5SyNr6bRQFq2I564paMvsGV2loUAH/v0wGwemD9DO5ouhbRlwkZwzWz+9L9kAHtz9RSFyymxf+mMXDv1yku/xxsGtpNLD4/9LI6OfgnNm1Wg/nuKiA32FsbgLxNvV2WOm5Udnxulq6m5jm5rbHi4hJ2EZV8X38855qvGoe12feeSqN1iYjV91dhsmsz+Jo3dFHIlJj0RaTmqhVjsuZQ1KpadF2J26nu6eP6tj4YwKeomTvXH479V2MEb/ib8u/1Xp4QW3D59H88EUM066vIrNf8KVyOqk7fGBivyRNy+EqwctkNHD+qAxsZu1n7+59dRXjJk8nwmICbsZg/C0wi6SMQmrKdNRYNcjUV5l456k08oa2Mfmy4Cxp0Snso1xStFlXb8uV4BNrjWDayAwMGm/UO3zjj/Q+zrATH6Opzso/bs1l13r9t03UG68HFj6ajtslmP27cozav673StgH/En9k1XLQqXXchJtnDJQu0bonTo3/tz+xGImTZ+N0fQZdzxVSGyimxf+lMWnixP83uYvlHy8KJG9m2xcfFsFqdn6a1l4rMI6h58Zb9VNKzYl+I3PS6Si0cHOcu0aYHS98cfFb54o5M3H01n2rxQObItk9u/KiVQ7c49q76ZIVr6eRP7ZjZxwjn6bmhyLsL7DP2WgfpbVKaHhnGFpJMdYtB7Gz1htkmvuKWPW3Eq2fRfFP36dS/Ee/Y1TL5rrjSx8JJ2kdBcX3Vqh9XB8JmwDft/kKLLi1QYVxbcijAZmjMrAGqG/ZK8QcPpF9fz6f4twOQVP3J7DV+/FqxTPYTweWPBgBs0NRq65p0y3/WmPR9gG/In9tM+3KqEp3mbWvCfu0fQdbueu5w4yeFwrS55N5eX7M2lpDNtQ8DPL/pXMno02Lr29kuyBgWuVGGEUDM2I5dL8bExG//w8wvKn3Dc5ivQ4q9bDUEJYn+QoTh6gbU/co4mO83LDvFJm3VTJ9u+j+NucPuwsUKt41n0awxfvJHLKrDpOOLcxIOc0CEF+nwRuPLUfU0ekk53gv59DWAb8E/upTVaK/53QJ5HB6fptPC4EnH5xPbc/WUhktIcX/pTNkmdTcDp0+tbEzwp3Wlj8jzT6jWxl1k2BKUuREmNh9oQcTh2YEpA0YNgF/LwkGxlxKnevBMY5w9JI0eEk7qGyBzi445lCTplVx1fvJfD4zXkc2BZe74CrSyP4171ZxMR7uPbeMowBWL84JjeeKyfkkhobuO912AV8lbtXAinCaGDmmExd7MQ9GrNFctGvq5j7SDEup+CpO3JYOj85LO72m+uNzP9TFl6vYM5DxcQk+LengBBw2qAUJg9ODfgeoLAK+DmJNjLVyhwlwGKtEUwfndnjcspRNZVcetfV2DSodjloXCu/m3+AidMa+PztRP52Ux67NoTu34yjTfDifZnUV5u4YV4JqTn+3VxlMgjOH5nB+LwEv57nSHoV8IUQiUKIj4UQuzv+2+VVCCE8QogfOv4t7c05eyNfo2+yomTFR3LW0J71xD1x4bNkbilg4uvP+HlUXbPaJJfeXsnNjxUhgOd/n8O/H0ujuV7f71KOlaNN8OJfsijcaeXqP5TTd7h/C+AZhOC8kRkMTNNuXkfIXizCFUI8BtRKKR8RQvwBSJBS/qxFuxCiWUp5TFta8/PzZUFBwXGNa1dFE8s3lf3ka8kxFn4xMe+4nk9RfOWr3VUUHOi6ANdt00dhcv58GaDbbOGpZZv8PbQuOR2CVf9O5NPFiVhsXqZdW81J5zcEXaenwzntgn/9OYu9myO58nfljD/LvztphWifzxmeGefX87SfS6yTUuZ39VhvUzqzgFc7Pn4VuKCXz+c343PV3b2ivVMGJNP/COU8Xnp1FdsnT8dlaZ/Ec1msbD9zBi8u+CSQQ/wJs0Uy7foafvv8QbL6OXjn6TT+cVsu+7cG76Suo+3owb6xppKn77rapw1kThuUEpBg353eBvw0KWXnrXQ5kHaE46xCiAIhxBohxBFfFIQQczqOK6iq8t03O8Zq0vXyOCV8CCE4b0R6l/tAWpJScdqiMToduM0WjE4HTls0rYnalwBJz3Ny82PF/OJPpTTXG3nqjlwWPJhObUVwleNqrDXyzG9z2Ls5ktm/7frOfuXCZ9m/pYCVPkqpjc2NZ5xObji7/WkJIVYB6V08dM+hn0gppRDiSPmhPClliRCiH/CpEGKzlHLv4QdJKecD86E9pdPt6HtoTE68bvqPKkqE0cCsMZm8sbaIhrafThLa6mvYNH02m6ddzsgVbxKlozaFQsDYM5oZdmILn72VyGdvJbBldTSnXljPWZfXYovRdzG28gNm/nlvFi0NRn55fynDJ7b85PEfG8i0W71sEauXLcJktvDYcabUchNtnKajml29zeHvBM6QUpYJITKAz6WUg7v5f14Blkkp3z7acb7K4ZtNBm48tS8WU5AnHZWQU9vi5M3vi7C7/LsM0F/qq0yseDmJdZ/EYo3ycuZltZx6QT1mq/5qz2z9NoqFj6UTYZHcOK+EnEE/nytprKlk6fxH2bx6FS6HnQiLlZEnn8PMOb8n9jjeZcVFRjB7Qi6RAV6S688c/lLg2o6PrwXe7+LkCUIIS8fHycDJwLZenrfHhmfGqmCv6FJilJlZYzKJMAbnu8/4FDdX3l3BXc8fpO/wNpa/lMKD1/bl87cTcLTp45pcTsGSZ1N48b4sktJd3P5EYZfBHn7eQMbtdGC1RR9XsI8wCmaMzgx4sO9ObxNwjwCLhRA3AAeBywCEEPnAXCnljcBQ4AUhhJf2F5hHpJQBCfhCtKdzFEWvMuMjOX9UJkt/KMUbpGUrM/s6ufGBUvZvtfLRgiSWzk/h0zcTOO2iOiZNb9As1VOy18wb/5tOyV4rp15Yx4wbqrttPt7ZQGbitMtZs+LN4564nTwkVZc7rHuV0vEnX6R0+iTbuHBsto9Hpii+t72skY+2lodEqeL9W62sXJjEzoIozFYvE6Y0cNqF9SRnBqZjVEujgQ8XJLN6WRy2GA9X3FnB8JNauv8ffWRoRixTR3Q17RkYR0vpBNcU+zEana3u7pXgMDQjFqfby6c7KrUeSq/1HW7npodKKN1n5vO3E1i9LJ6v309g0NgWJp7fwIiTmjH5oad6W7OBb1fE8emb8bQ2GZgwpYyZc1oD+g4jwRbBmUN6tsFOCyEb8ONtEfRNjtJ6GIrSY6Nz4vFIyRc79bMypzcy+zm58u4Kzr+hmrUfxrHmgzgW/DWTyGgPIyY1M/q0ZgaNae02zXI0Uravvln7UfvzO9oMxCVvgqZrMEUMwxZzv+8uqBsmg2DaqAzMJv1WrAnZlE6T3cX4PFUGWQk+BQdq+Wp3tdbD8DmvB3aut7Hhsxi2fBuNvcWI2eKlz/A2Boxuo8+wNtJynUTHe47YPEbK9rX0JXut7N0Uyeavo6kuNWMwSCRvIL2PAht/8v/0ZlnlsThtUIpmNXIOFXYpnQijQRe72hTleOT3ab9RCbWgbzDC0BNaGXpCK25nJbs22NhRYGPvJhsrXv6xWUxkjIfENBdWmxerzYswgL3FQFuLgfoqEy0Npo7nkwwc08oZl9QxYlIzyByWzs9h8+qdP1tW6W+5iTbG5eo/hRySAT830aY2WilBLb9PIiajgc93VobERO7hTGbJsBNbGHZi+2Rqc72Rkj0WKorMVBSaaag2YW9tD/AejyAy2kNckpvsgQ4y+znI6tf+X2vUofl53y2rPBbWCCPnDk9D6LWn5SFCMuCrYK+EgjE58ZgMglXbK0Iy6B8qOt7D4PxWBue39up5fLWs8licPTSVGKsfZqH9ICRz+IoSSvZUNvHB5nLc3sD8rTbWVLLgoTu55p5/+P3uONgNy4xlynDtlmB2xZ87bRVF8bMBqTFcPD47YLs2fV08LFTFRkZwxuDgekFUd/iKEiTqWpws2VDys4JrvnJ48bBOgVrlEkyEgEvGZ5OdYNN6KD+j7vAVJQQkRJmZPSGX3ET/BJl7X13FuMnTieioxx9hsTLuzBncq2E9fr0an5egy2DfHRXwFSWIRJqNXDg2i7F+WALoy+JhoSw5xsKk/sndH6hDIblKR1FCmcEgOGNwKmmxVj7dUYnT7bvSAVqscgkmJkN7A5tgXQmocviKEsRqW5ws31RKdbNT66GEhTMGpzBWJ92rjkTl8BUlRCVGmbliQi4js9TOcn/rmxyl+2DfHRXwFSXIRRgNnD0sjVljMomy6KvhRqiwmY2cM+xILbuDhwr4ihIi+qVE84uJfRiQGq31UEKKEHDu8HSiLME/5akCvqKEkEizkRmjM5k+KkPd7fvI+LyEkCm1HvwvWYqi/MzAtBhyEm18uauKraWNWg8naGXGWzk5SJdgdkXd4StKiGqv4pjOpfnZJEebtR5O0LFGGJk6IgNDkC7B7IoK+IoS4rITbFx1Yh6nDUrWdTcmPRECpgxPIy4yOKpg9pT66StKGDAYBOPzErluUh+GZ8YesaOU0m5ivyT6pYTe5LcK+IoSRqIsJs4dns7sCblkJURqPRxd6p8azYl9Q7M9qgr4ihKG0mKtXJafw4zRGSTYQitt0RuJUWamBEn3quOhVukoShgbkBpD3+RoNpc08N2+GlqdHq2HpBmb2cisMZlYTKG7nFUFfEUJc0aDYExOPMMyYll3sI71hXU+LcgWDCKMglljsoi3hfZqJpXSURQFALPJwEn9k7j+5D6MzW3vpxsODEJw3sgM0uOsWg/F71TAVxTlJ2xmE2cMTuXak/swIisOQ4jms6F9+eWZQ1LpH4IrcrrSq4AvhLhUCLFVCOEVQnRZjrPjuKlCiJ1CiD1CiD/05pyKogRGrDWCc4alcc1JeQxJjwnJpZyTB6cyMjt8Ko329g5/C3AR8OWRDhBCGIFngPOAYcBsIcSwXp5XUZQASYgyc97IDK6emMfAtOiQCfyTh6QyOsf3ncP0rFeTtlLK7UB3S5gmAHuklPs6jn0DmAVs6825FUUJrORoC9NHZVLZZGfNvlr2VTWj0/5JRyUEnDE4lTFhFuwhMDn8LKDokM+LO772M0KIOUKIAiFEQVWVaq2mKHqUGmNl5uhMrjwxlwGpvr/jb6yp5Om7rvZLe8UIo2D6qMywDPbQg4AvhFglhNjSxb9Zvh6MlHK+lDJfSpmfkqIaJyuKnqXGWJkxOpOrTsxjsA9z/CsXPsv+LQWsfP0Z3zxhB5vZyCXjc8K6X0C3KR0p5dm9PEcJkHPI59kdX1MUJQSkxFiYNjKDif2SKDhQy47yJjzeY8/13D19FG6n47+fr162iNXLFmEyW3hs2aZejTErPpKpI9OJtYb3ruJApHS+BwYKIfoKIczAFcDSAJxXUZQASowyc+7wdK4/uQ/j8xKOuTLnva+uYtzk6URY2tfDR1isjDtzBvcu+OS4x2QQghP7JXLJ+OywD/bQ+2WZFwohioGTgOVCiI86vp4phFgBIKV0A7cCHwHbgcVSyq29G7aiKHoVY43gtEEp3HhqX04fnNLjEsOxSalYbdG4nQ5MZgtupwOrLZrYxONL76bEWLgkP5tJ/ZNDqqZ9b/R2lc4SYEkXXy8Fph3y+QpgRW/OpShKcLGYjIzLTWBsTjwHalrZXNLA/qoWvEdZ2tNUX8Ok6bOZOO1y1qx487gmbiPNRib1T2JEZpwK9IcRUqfrqvLz82VBQYHWw1AUxYeaHW52ljeys7yZika7T587LjKC0TnxjMiKDekCaN0RQqyTUna5EVYVT1MUJWCiLSbG5yUyPi+R+lYne6taKKptpaS+7bgKtkWajfRJsjEgNYb+KVEhW9bYV1TAVxRFE/E2M+PzzIzPS8DjlVQ3O6hqclDV7KCh1UWby0Ob04PHKzEaBBFGgTXCSGKUmcQoM2mxVtJjrSptcwxUwFcURXNGgyAt1kpabOhXrNSSqpapKIoSJlTAVxRFCRMq4CuKooQJFfAVRVHChAr4iqIoYUIFfEVRlDChAr6iKEqYUAFfURQlTKiAryiKEiZ0WzxNCFEFHOzFUyQD1T4aTrAIt2sOt+sFdc3hojfXnCel7LKmtG4Dfm8JIQqOVDEuVIXbNYfb9YK65nDhr2tWKR1FUZQwoQK+oihKmAjlgD9f6wFoINyuOdyuF9Q1hwu/XHPI5vAVRVGUnwrlO3xFURTlECrgK4qihImgDvhCiKlCiJ1CiD1CiD908bhFCPFmx+PfCSH6BH6UvtWDa75TCLFNCLFJCPGJECJPi3H6UnfXfMhxFwshpBAi6Jfw9eSahRCXdfystwoh/h3oMfpaD363c4UQGVI9BAAAA0ZJREFUnwkhNnT8fk/TYpy+IoR4SQhRKYTYcoTHhRDiyY7vxyYhxLhen1RKGZT/ACOwF+gHmIGNwLDDjrkFeL7j4yuAN7UedwCueTJg6/j45nC45o7jYoAvgTVAvtbjDsDPeSCwAUjo+DxV63EH4JrnAzd3fDwMOKD1uHt5zacB44AtR3h8GvABIICJwHe9PWcw3+FPAPZIKfdJKZ3AG8Csw46ZBbza8fHbwFkiuNvad3vNUsrPpJStHZ+uAbIDPEZf68nPGeAB4FHAHsjB+UlPrvlXwDNSyjoAKWVlgMfoaz25ZgnEdnwcB5QGcHw+J6X8Eqg9yiGzgAWy3RogXgiR0ZtzBnPAzwKKDvm8uONrXR4jpXQDDUBSQEbnHz255kPdQPsdQjDr9po73urmSCmXB3JgftSTn/MgYJAQ4hshxBohxNSAjc4/enLN9wNXCyGKgRXAbYEZmmaO9e+9W6ZeDUfRLSHE1UA+cLrWY/EnIYQBeBy4TuOhBJqJ9rTOGbS/i/tSCDFSSlmv6aj8azbwipTy70KIk4DXhBAjpJRerQcWLIL5Dr8EyDnk8+yOr3V5jBDCRPvbwJqAjM4/enLNCCHOBu4BZkopHQEam790d80xwAjgcyHEAdpznUuDfOK2Jz/nYmCplNIlpdwP7KL9BSBY9eSabwAWA0gpvwWstBcZC1U9+ns/FsEc8L8HBgoh+gohzLRPyi497JilwLUdH18CfCo7ZkOCVLfXLIQYC7xAe7AP9rwudHPNUsoGKWWylLKPlLIP7fMWM6WUBdoM1yd68rv9Hu139wghkmlP8ewL5CB9rCfXXAicBSCEGEp7wK8K6CgDaylwTcdqnYlAg5SyrDdPGLQpHSmlWwhxK/AR7TP8L0kptwoh5gEFUsqlwIu0v+3bQ/vkyBXajbj3enjNfwOigbc65qcLpZQzNRt0L/XwmkNKD6/5I+BcIcQ2wAP8TkoZtO9ee3jNdwH/FELcQfsE7nXBfAMnhFhE+4t2cse8xH1ABICU8nna5ymmAXuAVuD6Xp8ziL9fiqIoyjEI5pSOoiiKcgxUwFcURQkTKuAriqKECRXwFUVRwoQK+IqiKGFCBXxFUZQwoQK+oihKmPj/EMi6nYjT8t0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, axes = plt.subplots(1, 1, figsize=(6, 4))\n",
    "\n",
    "# plot the 12th CV fold\n",
    "num = 12 \n",
    "\n",
    "# plot the training data in black\n",
    "axes.plot(\n",
    "    cv_folds.train_X[num - 1].detach().numpy(), \n",
    "    cv_folds.train_Y[num - 1].detach().numpy(), \n",
    "    'k*'\n",
    ")\n",
    "\n",
    "# plot the test data in red\n",
    "axes.plot(\n",
    "    cv_folds.test_X[num - 1].detach().numpy(), \n",
    "    cv_folds.test_Y[num - 1].detach().numpy(), \n",
    "    'r*'\n",
    ")\n",
    "\n",
    "# plot posterior means as blue line\n",
    "axes.plot(plot_x[num - 1].numpy(), mean[num-1].numpy(), 'b')\n",
    "\n",
    "# shade between the lower and upper confidence bounds\n",
    "axes.fill_between(\n",
    "    plot_x[num - 1].numpy(), \n",
    "    lower[num - 1].numpy(), \n",
    "    upper[num - 1].numpy(), \n",
    "    alpha=0.5\n",
    ");"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
